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ABSTRACT 

Guidelines for judging English methods-course 
textbooks need to be developed and applied to the increasing 
quantities of texts being published. Objective data for evaluation 
may be obtained through several approaches: (1) key word in context, 
in which the frequency of a word is accepted as representative of the 
importance of a concept; (2) analogue, in which texts are checked for 
the presence or absence of key concepts; and (3) paradigm, in which a 
researcher projects an ideal methods text as a bench mark against 
which others . may be compared. Another system providing a rough 
instrument for judging the strengths and limitations of textbooks is 
the Hare system. Using this letter system, a list of 31 books 
• published between 1958 and 1968 was sent to 100 college English 

methods instructors who were asked to rank the books from 1 (highest) 
to 4. Sixty-one instructors responded. The results of this survey 
correlated well with earlier data obtained through the use of the 
other three evaluation systems, showing that Hook # s text is best for 
a general overview of English teaching; Loban, Ryan, and Squire for 
in-service teachers; Bernstein for highest student interest; Guth for 
scholarship; and i j rton and Simmons for application. (JM) 
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RESEARCH STRATEGIES FOR EVALUATING THE 
STRENGTHS AND LIMITATIONS OF ENGLISH METHODS TEXTBOOKS 

Edward R. Fagan 

Introduction 

Judging the strengths and limitations of textbooks used in 
English methods courses is a task which is best approached with 
the caution implied in the title of one of Kierkegaard’s books, 
namely with “Fear and Trembling, the Sickness Unto Death.” 

This caution is justified by the dearth of objective criteria for 
making judgments about methods texts and by the sentient rec- 
ognition of the trap baited with judgmental concepts such as 
strengths and limitations. But the burgeoning publication of 
methods texts requires that some evaluative criteria be devel- 
oped, and the old saw, “Let George do it” requires that somebody 
be “George,” even if George turns out to be Charlie Brown. With 
such tacit martyrdom in mind, strengths are defined for this 
study as those criteria which have a measurable impact on the 
methods used to teach English in contemporary English class- 
rooms. Measurable in this context means evidence of students’ 
increased academic awareness and/or behavioral change. Limita- 
tions are defined as those incomplete recommendations or out- 
right gaps within methods textbooks which have a high proba- 
bility of distorting the prospective English teacher’s perceptions 
of his professional duties within his discipline. 

These arbitrary and judgmental definitions, if accepted, per- 
mit this generalization: no single textbook in any given methods 
course is sufficient to provide prospectve English teachers with 
the contexts they need to guide students’ apprehension of Eng- 
lish as English is defined in the research literature extant at 
this time. 1 Notice the time factor in the generalization. It is 
possible that some linotype system is even now printing out a 
refutation of the generalization, but for methods textbooks pub- 
lished up to and including January 1968 the generalization holds. 
English methods course instructors are well aware of this gen- 
eralization, but it needs to be stated for some of our colleagues 
in other disciplines and for all critics of English methods courses. 
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this latter group exacting the stance that there is no content to 
methods and/or that methods deal with common sense, defined as 
the stuff that everyone knows.” That the contents of textbooks 
U Je Jt° k 30 * 1 English methods have grown exponentially in both 
affective and cognitive domains is well documented by the mass 
of data stored in computers at English and/or Educational Re- 
sources Information Centers (ERIC) throughout the country. 2 

consequence of the ERIC phenomena is that contrary to 
critics statements that methods texts are the stuff “everyone 
knows,” just the opposite is the case ; multiple methods texts are 
necessary to provide even a modicum of what is known about 
teaching English in today's high schools. Most English methods 
instructors who use textbooks are well aware of this fact, for of 
the 61 colleges and universities reporting undergraduate Eng- 
hsh methods course in their programs, 2 none used a single text, 
and all used additional materials to supplement the multiple 
texts used. Variety in the selection of methods textbooks was 
another characteristic^ of the 61 colleges and universities re- 
spondting to the questionnaire and such diversity implies that 
the strengths and limitations of textbooks used to teach methods 
are determined by each instructor. Yet such solecism can be 
strengthened by objective criteria even though such criteria 
lack the empirical rigor of the logical positivists. What follows 
is a description of techniques to develop more objectve guide- 
lines for the selection of textbooks used to teach English methods. 
Procedures 



Criteria used to judge the strengths and limitations of text- 
books commonly used in English methods courses can be derived 
• om i evaluations of the English teacher's profes- 

sional duties. These duties are described, albeit inferentially, in 
tobhoj^apluca 1 references such as the “English,” “Literature” 
and Teacher Education” sections of the Encyclopedia of Edu- 
caUonal Research; Volumes II, III, and V of the National Coun- 

u Arts series ’* in references as recent as the Octo- 

ber 1967 edition of PMLA * and in similar reports. Asking the 
question : what textbook contents are necessary to teach Eng- 
lish aecording to these sources?” generates answers which 
might ^be regarded as criteria which every English teacher 
should know. But such criteria would be inefficient, paradoxi- 
cal *? d overgeneralized— world, in fact, duplicate the system 
usedby Charters and Waples in their monumental teacher train- 
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j . ^ ^ til© winnowing process, the survey 

was limited to post 1958, using as a rationale the fact that major 
curricular reorganizations were initiated at that time. But 
even m the decade 1958-1968 the wealth of new material to be 
searched would be unmanageable unless the system were modi- 
fied still further. Such modification might be accomplished by 
using a system known as key word in context technique. 6 Such 
a system popularized by humanists who use the computer for lit- 
erary analysis, requires that the frequency of a word in context 
be accepted as one clue to the importance of that word as an 
wentifier of a value-loaded concept for the context involved. In 
ifinglish, for example, notice the frequency and the context of 
key words such as: “language,” ‘literature” and “composition” 
since 1958. That this system has validity can be documented by 
the fact that key themes in Hamlet were identified by its use. 

he point, however, is that this system was used, along with 
others, to identify the strengths and weakness of textbook used 
m teaching English methods courses. 

One of these other systems might be called the analogue sys- 
tem, where the analogue search operates upon the principle that 
it is reasonable to adapt compatible systems from other domains 
to the analysis of textbooks used in English methods courses. 
Specifically , Lynch and Evans performed a monumental study of 
textbooks used to teach English in high schools.* Their system 
might be described as new critic in the sense that each textbook 
was examined according to its potential for achieving the ob- 
jectives of a good English program. A “good English program,” 
Lynch and Evans imply, is one which displays contents and 
processes m such a way that students have a high probability of 
understanding concepts and principles related to literary genres, 
language and composition. Although Lynch and Evans* arbitrary 
judgments about the worth of myriad textbook series is based 
on a prim criteria, their system for analysis does have face 
validity. When, for example, they score the deadly repetition of 
diagramming, they support their criticisms by overwhelming 
examples from text after text and series after series until read- 
a that diagramming is a ritualistic exercise in 

* U j in ^ther textbook analysts 8 use systems similar to Lynch 
and Evans, but the point is that such systems provide analogi- 
cal models for judging the strengths and limitations of textbooks 
used m English methods courses. 
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®Jl n f not ^ er system for textbook evaluation is that cavalierly 
identified as the paradigm. To assuage the violence done to pro- 
fessional paradigm makers, the adverb, cavalierly is used, but 
me system behind the concept, paradigm, for textbook evalua- 
tion uses Max Weber’s “ideal-type” » paradigm. Weber’s para- 
mgm attempted to gauge the religio-economic behavior of the 
collective, economic man. His construct, the “ideal type,” en- 
f hypothetical religio-economic man with ideal economic 
attributes. Then this idealized economic paragon served as a 
benchmark against which man’s economic behavior in real eco- 
nomic situations was measured. 

. W ^ er>s concept might apply equally well to ascertain the 
strengths and weaknesses of textbooks used in English methods 
In * hls c^e, an ideal methods textbook is projected. 
f ?5 ^ contents of this ideal methods textbook are 
the two previously described techniques, namely bib- 
liographical and analogical searches of relevant literature. These 
two techniques would permit a researcher to generate a list of 
concepts and/or behaviors which, ideally, every prospective Eng- 
lish teacher ought to experience. But such a book, if conscien- 

JS-Wjjf probably have more pages than Gone 
e Wind. What are some of the alternatives to such an 
economicaUy unfeasible tome? One alternative is to narrow the 
scope of the methods books to, say, the recommended tripod: 
language, literature and composition. And, in fact, many con- 
temporary methods books do just that. Another alternative is to 
create a senes of books or a collection of articles related to vari- 
ous aspects of the language arts and mass media. And other 
methods textbooks use that alternative. Still another alternative 
is to identify some specialty within English as a field— the dis- 
vi the ® 1 ® mentar y language arts program, linguistics 
• i^ he ^ lke ~ and to pubIish a detailed explication to the spe- 
cialty. Examples of these and other alternatives to the monu- 

r t^ maTket mPlied ^ ^ type ” »***&* are Presently 

. ^ bese textbooks and their subsequent ranking accord- 

• J^ e J! ystem 10 can provide a very rough instru- 
ment for Judging the strengths and limitations of textbooks used . 
m English methods courses. The Hare system (named after the 
man who invented it) asks the user to rank in order, 1 - n, any 
given list of data, where 1 stands for the highest rank and other 
numbers correspondingly lower ranks. Numerals used do not 



78 



THE ENGLISH RECORD 



ERIC 



have to he defined but, for clarity, were defined for this survey 
as follows: 1. “Most valuable of the methods texts I used”; 
2. “Valuable but contains minor ellipses” ; 3. “Generally valuable, 
but needs supplementary materials”; 4. “Minimally valuable, 
needs massive supplementation.” These numerals were to be 
placed after a list of English methods textbooks which had been 
published between 1958 and 1968 and which were on recom- 
mended book lists distributed by NOTE, MLA, ASCD, CEE and 
the like ; as well as on lists about methods textbooks supplied by 
publishers, book review editors, .and/or authors of research 
articles. This list of 31 books was then sent to 100 English 
methods instructors at colleges and universities selected from 
the Council of Graduate Schools (CGS) list. The selection proc- 
ess attempted to include state universities, liberal arts colleges 
with teacher education programs, teachers colleges, and private 
and parochial institutions with teacher education programs. 
Returns from 61 of the 100 colleges contacted revealed the fol- 
lowing ranking of methods texts currently extant in the field. 



Number of Colleges 



Author/Title 


Reporting 


Hare Rating 


Hook, The Tchng. of H.S. Engl. 


45 


1.87 


Loban et al. Tchng. Lang, and Lit. 


35 


1.97 


Fowler, Tchng. Lang., Comp., 
and Lit. 


30 


2.07 


Guth, Engl. Today and Tomorrow 


28 


2.07 


Lewis/ Sisk, Tchng. Engl. 7-12 


24 


2.17 


Sauer, Engl, in Sec. School 


34 


2.41 


Wolfe, Creative Ways to Tch Engl. 


33 


2.55 


Burton & Simmons, Tchng. Engl, 
in Today's High Schools 


29 


2.41 


NCTE, Lang. Arts in See. School 


27 


2.67 


Bernstein, Tchng. Engl, in H.S. 


35 


2.71 



All 61 colleges ranked one or more of the 31 textbooks listed 
and many colleges went on to list additional texts in a category, 
“other,” provided on the ranking sheet. Arbitrarily, a cut-off 
point of 24 identifications was set as the lower limit for reporting 
textbook popularity and, implicitly, their virtues. Basis for such 
a decision was that the universe (100 colleges) was so small that 
any textbook not ranked by at least one-third of the 100 colleges 
would be questionable as representative of methods textbook 
popularity. 

Results 

Uses of the three search systems described previously (key 
word in context (KWIK) ^/bibliographical, analogue, and ideal 
type) to evaluate the strengths and limitations of English meth- 
ods textbooks provides data which correlate well with the survey 
results noted above. Using bibliographical references and KWIK 
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techniques, for example, generated methods concepts such as: 
reading, writing, speaking listening, literature, language com- 
position, structure, technique, philosophy, linguistics and the 
like. These key words serve to identify the parameters for the 
analogue system of evaluaton whereby texts are checked for the 
presence or absence of these key concepts in methods textbooks 
an “ideal type” can be derivd. This ideal type, in turn, can serve 
as a paradigm against which the contents of any methods text- 
books can be measured. 

Using the ideal type as a benchmark all of the above text- 
books have strengths such as : techniques, philosophies, contents 
and applications of same for the conduct of English teaching; by 
the same benchmark all can be faulted for their omission of 
technology, research, individualized instruction, deep structure 
analysis, the disadvantaged as these concepts impinge upon the 
teaching of English. One fault common to all of these textbooks 
is the inept use of thought questions, that is, questions which 
stretch students* minds, involve the class as a whole and further 
the systematic development of concept building. Too often illus- 
trative questions can be answered “yes” or “no” and thereby en- 
gage the teacher and student in a one-to-one dialogue to the 

detriment of the rest of the class. Or the questions are so framed 
that answers are implicit in their framing. Bernstein’s methods 
textbook is regarded by some students 12 as most valuable 
for prospective teachers because of its focus on the questioning 
technique as a fulcrum for heuristic learning. 

Conclusions 

Each of the ten listed textbooks and the others which com- 
prised the 31 of the total sample has its strengths and limita- 
tions, but how strengths and limitations are defined is a func- 
tion of each instructor. Since all instructors tend to supple- 
ment the textbook used with other materials and/or text- 
books, one would be foolish to make judgments about a mythi- 
cal “best” textbook apart from an ideal type. “Best for what 
purpose?” would be the question with myriad answers. For 
a general overview of English teaching perhaps Hook, for 
in-service teachers Loban, Ryan and Squire (but for pre-service 
teachers Loban et al. is overwhelming) for highest student 
interest Bernstein (but the contents are pretty traditional 
seemingly based on the New York City English syllabus) for 
scholarship Guth (but after Guth one wonders whether stu- 
dents would be prepared to work with the disadvantaged) for 
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application. Burton and Simmons (but necessary “how-to” de- 
tails are missing from their collection of readings). Supporting 
details for judgments about these and other methods textbooks 
are beyond the scope of this report. (Such detail would require a 
400 page tome similar to the Lynch and Evans study.) But the 
research strategies previously suggested, if employed to evalu- 
ate the strengths and limitations of current English methods 
textbooks, would help generate unique and valuable blueprints 
for the methods textbooks of future. 
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